1. Field of the Invention
The present invention relates to an image processing apparatus for decoding a stream of encoded image data and outputting decoded image data and a method for controlling the same.
2. Description of the Related Art
Encoding of image data is performed as, for example, a process of efficiently storing image data captured by an imaging apparatus, such as a digital camcorder, on a storage medium, such as a digital versatile disc (DVD), or a process of reducing, when transmitting image data via broadcast waves, the amount of data while keeping the image quality by using redundancy unique to the image data.
Streams generated by encoding image data are decoded by, for example, an image processing apparatus shown in FIG. 6.
An image processing apparatus 200 includes, as shown in FIG. 6, an input unit 210 that receives input streams of encoded image data from the outside of the image processing apparatus 200, a controller 220 that analyzes the input streams, a decoding processor 230 that decodes the input streams, a memory 240 that temporarily stores the decoded image data, and a renderer 250 that reads the image data from the memory 240 and outputs the image data to the outside of the image processing apparatus 200.
The input unit 210 receives an input stream of image data encoded in conformance with an encoding standard, such as the Moving Picture Experts Group (MPEG), and supplies the input stream to the controller 220 and the decoding processor 230.
The controller 220 analyzes the syntax of the input stream which has been received at the input unit 210 and supplies, of information obtained from the analysis result, information on the image size of image data corresponding to the input stream as control information to the memory 240.
The decoding processor 230 decodes the input stream received at the input unit 210 and writes the decoded image data to the memory 240 sequentially on a picture-by-picture basis.
The memory 240 stores the image data supplied from the decoding processor 230 in accordance with a memory map indicating which data should be stored at which storage position. More specifically, the memory 240 sets a memory map in accordance with the information on the image size, which is supplied from the controller 220, and stores data of pictures written by the decoding processor 230 in storage areas.
The renderer 250 reads the decoded image data obtained on a picture-by-picture basis from the memory 240 in accordance with the display order of the image data and outputs the image data to the outside of the image processing apparatus 200.
The image processing apparatus 200 with the foregoing structure performs a process shown in the flowchart of FIG. 7 in order to decode a stream of image data whose image size changes before the end of the stream and stores the decoded image data in the memory 240.
In step S101, the input unit 210 determines whether an input stream to be decoded has been received. When an input stream has been received, the flow proceeds to step S102. When no input stream has been received, the process is terminated.
In step S102, the controller 220 analyzes the input stream received at the input unit 210, and the flow proceeds to step S103.
In step S103, the controller 220 determines whether the image size of image data in the input stream received at the input unit 210 changes on a picture-by-picture basis in accordance with information analyzed in step S102. When the image size changes, the flow proceeds to step S104. When the image size does not change, the flow proceeds to step S105.
In step S104, the controller 220 supplies control information indicating that the image size changes to the memory 240. The memory 240 switches a memory map in accordance with the control information, and the flow proceeds to step S105.
In step S105, the decoding processor 230 decodes the input stream received at the input unit 210 and writes the decoded image data to the memory 240 sequentially on a picture-by-picture basis, and the flow returns to step S101.
In the image processing apparatus 200 which performs the decoding process in accordance with the foregoing steps, for example, as shown in FIG. 8, decoded image data of each picture obtained by the decoding processor 230 is supplied to the memory 240 in synchronization with a write timing. In synchronization with a read timing which is delayed relative to the write timing by, for example, a time corresponding to one picture, the renderer 250 reads the image data stored in the memory 240. In the specific example illustrated in FIG. 8, a stream before the image size change is called an “input stream A”, and a stream after the image size change is called an “input stream B”. The input streams are assumed to consist solely of intra coded (I) pictures and predicted (P) pictures each of which performs no backward prediction with reference to the following picture in the display order.
In the image processing apparatus 200, this delay in timing is taken into consideration. After the renderer 250 reads from the memory 240 image data of a picture displayed at last in the input stream A, decoding of the input stream B starts, including switching of the memory map. This is performed to prevent the image data of the picture displayed at last in the input stream A from being overwritten by image data in the input stream B, which is written on the basis of the switched memory map.
Therefore, in the image processing apparatus 200, there is a blanking period in which the renderer 250 is incapable of outputting image data of pictures, which are arranged in the display order, to the outside at the time the image size changes. In the specific example shown in FIG. 4, a blanking period taking a display time corresponding to one picture occurs at the time the memory map is switched. Therefore, a display apparatus that displays a video image represented by image data output from the image processing apparatus 200 is incapable of displaying a video image without interruption, that is, without involving the foregoing blanking period at the time the image size changes.
The image processing apparatus 200 can perform the decoding process while avoiding the occurrence of a blanking period by increasing the storage capacity of the memory 240 as in the following exemplary manner, without changing the memory map at the time the image size changes.
For example, when encoding is performed in the Advanced Video Coding (AVC), level 4.0, 4:2:0 format, decoding of encoded streams which have a standard definition (SD) size (image size of 720×480 pixels indicating luminance) and a high definition (HD) size (image size of 1920×1088 pixels indicating luminance) involves the following numbers of memory faces in the memory 240. The number of memory faces indicates the number of storage areas for storing pictures.
When it is assumed that the storable capacity of the memory 240 before the change is 12288×1024 bytes, the number of memory faces corresponding to SD-sized image data is approximately 16 (12299×1024 [bytes]/720×480×1.5 [bytes]). The number of memory faces corresponding to HD-sized image data is 4 (12299×1024 [bytes]/1920×1088×1.5 [bytes]).
Accordingly, the memory size in which items of image data can be sequentially written but not overwritten without switching the memory map is a memory size ensuring the image size and the number of frame faces satisfying both cases (SD size and HD size). More specifically, the memory size is 50135040 bytes (1920×1088×1.5×16). This memory size is four times larger than the memory size before the change at which the memory map is switched.
Therefore, in order to enable the image processing apparatus 200 to perform the decoding process without switching the memory map at the time the image size changes and without involving any blanking period, it is necessary to significantly increase the memory size, resulting in an unavoidable increase in the cost.
Japanese Unexamined Patent Application Publication No. 2003-58144 describes, in order to reduce the processing load on an image processing device and to prevent image quality deterioration of a display image, an image display control apparatus for alternately storing items of generated display data in first and second memory sections, sequentially reading the items of display data in the display order, and displaying a display image.